A Canonical Form of Arithmetic and Conditional Expressions

When implementing a software, developers express conceptual knowledge (e.g. about a specific feature) not only in program language syntax and semantics but also in linguistic information stored in identifiers (e.g. method or class names) [6]. Based on this habit, Natural Language Program Analysis (NLPA) is used to improve many different areas in software engineering such as code recommendations or program analysis [7]. Simplified, NLPA algorithms collect identifier names and apply term processing such as camel case splitting (i.e. “MyIdentifier” to “My” and “Identifier”) or stemming (i.e. “records” to “record”) to subsequently perform further analyzes [10]. In our research context, we search for code locations sharing similar terms to link them with each other. In such types of analysis, filtering stop words is essential to reduce the number of useless links. Just collecting, splitting, and stemming the identifier names, can result in a list of terms with divergent grade of usefulness. For example, the terms “get” and “set” are used in most Java application due to common coding practices and not to express any conceptual knowledge. These terms corrupt the program analysis leading to unreasonable findings. To reduce this noise, a typical approach in natural language processing is to filter terms known as useless (aka “stop words”). For natural languages, many stop word lists are publicly available. However, as Høst et al. [5] identified, developers use a more specific vocabulary than in general spoken language. So common stop word lists are not reasonable to be used in program analysis, they even depend on domain, application type, developing company, and project settings. In this paper, we propose an approach to develop reusable stop word lists to improve NLPA. We i) propose to distinguish different scopes a stop word list applies to (i.e. programming language, technology, and domain) and ii) recommend types of sources for terms to include. Our approach is closely related to the work of Ratiu [8], who recommended considering domain knowledge for program analysis in general. We propose a specific application of the concept as guidelines